Validating Cross-Cultural Studies of Personality p. 1 of 19 What do Cross-National Comparisons of Personality Traits Tell Us? The Case of Conscientiousness

Much research contrasts self-reported personality traits across cultures. We submit that this enterprise is weakened by significant methodological problems, particularly the reference-group effect, undermining the validity of average country scores of personality traits. Behavioral and demographic predictors of conscientiousness were correlated with different cross-national measures of conscientiousness based on self-reports, observer-reports, and perceptions of national character. These predictors correlated strongly with perceptions of national character, but not with self-reports and peer-reports. Country-level selfand peer-report measures of conscientiousness failed as markers of between-nation differences in personality. Validating Cross-Cultural Studies of Personality p. 3 of 19 Canadians and Americans both agree that Canadians are generally more agreeable than their southern neighbors. Is this judgment accurate? One way to investigate this question would be to compare perceptions of the average personality of Canadians and Americans with the actual average reported personality traits obtained from people in these countries. In an influential analysis (McCrae & Terracciano, 2006; Terracciano et al., 2005), there was no correlation between profiles of actual reported personality traits and “perceptions of national character” (PNC). On this basis, McCrae and Terracciano (2006) argued that perceptions of national character are illusory. We found these results surprising (also see McGrath & Goldberg, 2006). Past research has shown that agreement on cultural differences becomes stronger with increased cultural contact (Triandis & Vassiliou, 1967), and that perceptions of group differences are often, although not always, quite accurate (Jussim, 2005; McCauley, 1995). Conceptions of culture as a “shared meaning system” suggest that we can make meaningful and relatively accurate judgments about our own culture (Wan et al., 2007). The assessment of individual differences in personality that rely on selfand peerreports has become a widely used method of investigating cross-cultural differences (e.g., McCrae & Terracciano, 2006; Schmitt et al., 2007). Therefore, determining the validity of this method across cultures is of utmost importance (Norenzayan & Heine, 2005). In what follows, we investigate the validity of self/peer-reports and PNC comparisons across cultures. We argue that methodological problems compromise the validity of aggregate self/peer-report data across cultures. Focusing on conscientiousness as a test case, we present new behavioral and demographic evidence that suggests that the PNC Validating Cross-Cultural Studies of Personality p. 4 of 19 data, not the aggregate self-report data, are a more valid measure of between-culture differences in behavior. Comparing means on subjective Likert self-report scales across cultures is the most commonly used method for investigating cross-cultural differences, yet there are many methodological challenges associated with it (for discussion, see Cohen, in press; van de Vijver & Leung, 2001). One such challenge is the reference-group effect (RGE; Heine, Lehman, Peng, & Greenholtz, 2002; Peng, Nisbett, & Wong, 1997), the tendency for people to respond to subjective self-report items by comparing themselves with implicit standards from their culture (Heine et al., 2002). Consider how one would respond to the items on the Revised NEO Personality Inventory (NEO-PI-R; Costa & McCrae, 1992), for example, the item “I am not a very methodical person.” One’s answer greatly hinges on one’s understanding of the norms for being methodical. Because norms differ across cultures, the RGE systematically distorts cultural differences. Importantly, the RGE is only a problem for comparing aggregate data between cultures – it is not implicated in contrasts of individuals within a shared cultural context. The RGE is an issue in self/peer-reports, as respondents need to compare their evaluations against standards largely based on observations of people within their cultures. However, the RGE is less likely to contaminate perceptions of national character. This is because when people are asked to evaluate their “average compatriot” they would likely bring to mind a standard that lies outside their own culture, for example, a perceived international norm. Some evidence for this is that the variance of the PNC cultural means was much larger than the variance of the cultural means of selfor peerratings (Terracciano et al., 2005). Validating Cross-Cultural Studies of Personality p. 5 of 19 Establishing Validity: Behavioral and Demographic Predictors of Personality Across Cultures What is the existing evidence that cross-national comparisons of personality traits are valid? One source is that some of Hofstede’s (2001) cultural dimensions correlate with some of the Big Five measures (McCrae, 2002). However, these correlations are not a priori theoretically relevant (e.g., neuroticism and masculinity are correlated, but for theoretically unknown reasons), and we note that there are zero significant correlations between any of the Big Five and Hostede’s five dimensions that replicate across three independent measures of the Big Five (McCrae, 2002; McCrae et al., 2005; Schmitt et al., 2007). Another source of evidence is that geographically and historically related cultures (such as Germany and Austria) have similar personality profiles (McCrae & Terraciano, 2006). However, cultures with similar reference-groups would also yield this same pattern. Other criteria are needed to validate cross-national comparisons of personality traits. We focus here on a criterion that is frequently relied upon in evaluating trait measures (e.g., Gosling, Ko, Mannarelli, & Morris, 2002): whether the trait-scores predict actual behavior.